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ABSTRACT 

It is common to see students multitasking or switching between different tasks on the computer while also listening to the 
teacher lecture in the front of a classroom. In today’s classrooms, students have much greater control over how they use 
their time, with the classroom integration of computers and mobile devices combined with social media and text-based 
chat tools. It is important to understand how students learn with the connected technologies. This study examines high- 
school students’ ability to take notes and obtain information from a video while simultaneously carrying a conversation 
online with a friend. 
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1. INTRODUCTION 

This study examines issues related to media multitasking and learning in educational environments. The rise 
of connected devices in the classroom has increased the level of multitasking during lectures and lessons. 
Students are often observed taking notes while texting a friend on their phone or updating their Facebook 
page. Students can be observed performing these types of dual-tasks throughout the day and it has become a 
natural part of how they function. What is the cognitive cost of this practice? 

Poldrack and Foerde (2007) found that people had a harder time learning new things when their brains 
were distracted by another activity. The Functional Magnetic Resonance Images (fMRIs) used by researchers 
showed that when people learned without disttaction, an area of the brain known as the hippocampus was 
involved. This region of the brain is critical to the processing and storing of information. However, the 
hippocampus was not engaged when people learned while multitasking. Instead, the region of the brain called 
the sttiatum was activated. The striatum is activated by stimuli associated with reward or by stimuli 
associated with aversive, novel, unexpected or intense experiences (Schultz, 2010). Results indicated that 
learning while distracted or multitasking would alter the brain’s learning processes and change the way 
people learn (Poldrack & Foerde, 2007). Foerde, Knowlton, and Poldrack (2006) found that learning new 
things is dependent on working memory where as habit learning is not as sensitive to working memory. Some 
tasks such as learning new skills may require high cognitive loads, while other familiar and automatic tasks 
may require lower cognitive loads. Lin and Bigenho (2012) examined undergraduate student’s memory recall 
under nine conditions in a 3x3 study with three levels of environmental distraction and three levels of 
note-taking. The study found significant interactions between environment and method of note-taking as the 
computer seemed to mediate the effect of an auditory distraction during computer aided note taking. The 
study also found that students were more successful when taking notes at lower levels of distraction and that 
those who did not take notes consistently performed poorly on the recall tasks regardless of distraction level. 
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2. THE STUDY 

This study was designed to examine high-school students’ abilities to take notes and obtain information of a 
video while carrying a conversation online with a friend at the same time. The experiments were designed to 
imitate the practice of chatting online with friends while attending a classroom lecture. Two scenarios were 
created: one was to have students watch a video and take notes of the video only; another was to have 
students watch a video and take notes of the video while chatting with a friend. Specifically, students 
observed a 20-minute video lesson and were told they would be tested on the content of the video after each 
lesson in both scenarios. The students were also encouraged to take notes following a format and method 
most comfortable to them. The difference lies in the online chat: in one scenario, the students focus on the 
video and take notes only; in another scenario, the students maintain a chat with an Artificially Intelligent 
(AI) agent while observing the video and taking notes. Two videos of the same length and difficulty level 
were used. All students participated in both scenarios. In addition, four conditions were created so that the 
sequence of the videos and the sequence of the chat/no chat situations would not affect the results of the 
study. Data were collected from the student participants’ quiz scores of the videos, their notes, and their chat 
transcripts. The participants were also asked to fill out an open-ended questionnaire to describe their 
experiences with the experiment and their learning habits. The study employed both quantitative and 
qualitative measures following a repeated measures design. 

Questions we hoped to answer through this approach include: 

• What were the patterns, if any, to the distribution of notes as compared to the presentation of the 
video lecture and the level of chat activity during the chat condition? 

• What were the patterns, if any, in the answers on the quizzes as related to the flow of the 
lectures, the notes that were taken, and the level of chat activity in the chat condition? 

• How might the lecturer’s style and method of delivery affect the notes recorded, the level of chat 
activity or the participant’s ability to recall information? 

• What does this indicate about the nature of the topics and the methods of delivery of the 
information in increasingly complex and connected classrooms and meeting centers? 

2.1 Participants 

Participants in this study were from a small to mid-sized independent high school in the Southwest United 
States. An a priori analysis was conducted from the initial study and it was determined that 40 to 54 
participants would allow us to detect a medium effect of f = 0.25 with the alpha level set at p < .05. A total of 
39 students participated in the study. They ranged in age from 15 to 18 with the average being 17 years. All 
participants were in 9 th , 10 th , 11 th , or 12 th grade. Participant consent and assent forms were obtained for each 
participant. 

The experimental conditions, data sources and design are summarized in table 1. This format allows for 
the following comparisons: video presentation order (First::Second); video topic (Video Topic 1:: Video 
Topic 2), and Chat (Chat::No Chat). Each participant was assigned to a condition and completed all tasks 
related to that condition. There are two scenarios (Chat/No Chat) which are presented in a balanced format to 
help identify any effects of video content and presentation order. 


Table 1. Sequence and Number of Participants in Each Condition 


Conditions First video and quiz 

Second video and quiz 

Survey 

Number of participants 

Condition I Video A (No Chat) 

Video B (Chat) 

Post Survey 

10 

Condition I Video A (Chat) 

Video B (No Chat) 

Post Survey 

10 

Condition ( Video B (No Chat) 

Video A (Chat) 

Post Survey 

10 

Condition I Video B (Chat) 

Video A (No Chat) 

Post Survey 

10 


2.2 Data Analysis 

The study generated several different artifacts as data and consisted of both qualitative and quantitative data 
forms. Additionally, the video lessons provided a time code that could be aligned with each data type 
allowing for all data to be examined across the temporal continuum of the video lesson. 
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Video Transcript: Each video transcript was organized into sentences with the video on happiness 
resulting in 256 sentences compared to 184 for the video on energy. This allowed the note and quiz data to be 
mapped directly back to the video transcript using what we called knowledge units. Knowledge units are a 
count of specific mappings from the notes to specific sentences. 

Notes of the videos: Participants took notes during each of the video lectures recording content from the 
video lectures. Notes were taken on a computer or on paper as decided by each participant. Notes were coded 
according to the knowledge units they contained. Each knowledge unit was mapped to specific sentences 
from the transcript providing a temporal scale to the notes in relation to the content of the video. Data were 
then loaded into Atlas. TI for future analysis of patterns. A scaled score for comparison between videos was 
created by dividing the total number of knowledge units recorded for each set of notes by the total number of 
knowledge units (sentences) in each transcript. This value was multiplied by 100 resulting in a scaled score. 

Post video lecture quiz: At the end of each video lecture, the participants were given two minutes to 
review their notes and then were asked to take a short quiz on the material from the lecture. The questions 
were directly related to material presented in the lecture they had just observed. The quiz results were 
subjected to an item analysis and then the results were mapped back to the video’s temporal scale and 
compared with the analysis of the temporal analysis of the notes taken. 

Chat transcript: Chatting with the AI chatbot created a transcript of the exchange. Each chat transcript 
was archived preserving the content of the conversation. Each transcript has a time code embedded and all 
chats were all initiated with the same method so that they were aligned with the time-code of the video 
lecture. The units of analysis for the chat were the individual transmissions and frequency of transmission 
mapped to the video temporal scale. Chats were also analyzed for content and any emerging trends. 

Post experimental survey: This was an open-ended questionnaire that all participants completed at the end 
of each experiment. The survey asked the participants about their experience during the two experimental 
trials. There was no timestamp for this data. The post survey results were subjected to content analysis using 
the word as the unit of analysis. 


2.3 Result 

Using a paired t test, the analysis of the chat/nochat conditions indicated significant differences for quiz 
scores in the two conditions t(37) = -5.12, p<0.001 as well as significant differences for knowledge units 
recorded in notes t(37) = -4.02, p<0.001. Additionally, the analysis of knowledge units recorded for each 
video topic also indicated significant differences t(37) = 3.53, p=.001. There are no significant differences for 
the presentation order of videos or chat. Additionally, quiz results based on video topic indicated no 
significant differences. 

Table 2. Table of means for comparisons. Items reaching levels of significance using paired t-tests are indicated with **. 


Data Type 

Comparison Condition Mean S.D. 

Quiz Scores 

w . , . Happy Video 11.28 2.33 

Video Topic Energy Video 10.74 2.28 

w . , First Video 11.39 2.07 

Video Order Second Video 10.64 2.49 

Chat Condition** j? 3 * ^ ^ 

No Chat 12.15 1.59 

Mapped 

Knowledge Units 

Video Topic** Happy Video 9.75 5.53 

video lopic Energy Video 13.75 6.40 

.... „ . First Video 13.12 6.06 

Video Order Second Video 10.41 6.28 

Chat Condition** ^ ^ 

No Chat 13.95 6.39 


Comments from the post experimental survey indicated elevated intrinsic interest in the happiness video 
compared to the energy video while the number of knowledge units recorded in participant notes was higher 
for the energy video . This may have been linked to less familiarity or comfort with the energy topic and 
therefore the perceived need to record more notes. However, since the difference in quiz scores between the 
two video topics did not reach significance, we can treat them as equal when comparing them in the two chat 
conditions. Additionally, we can ignore video order as it did not reach significance. 
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Therefore, when students were in the chat condition, they performed significantly worse on the quiz 
immediately following the lesson, indicating that chatting dining the lecture had a significant effect on their 
ability to code the information and commit the information to short-term memory. Since students were also 
allowed two minutes to review their notes prior to testing, it would seem logical to believe that their notes 
were also significantly different between the chat and no chat condition. This conclusion gets increased 
support when examining knowledge units mapped from notes taken under each chat condition. Students 
recorded more knowledge units when not in the chat condition than they did when chatting. 

The large standard deviations in the findings related to mapped knowledge units were likely the result of 
some students taking minimal notes during the lecture and not taking notes at all during the chat condition 
even though they were instructed to do so. Some initial qualitative analyses of the notes and chat transcripts 
indicated that: 1) there were differences in the use of language between notes taken when chatting and 
without chatting, 2) there were cases of chat transaction ending up in notes, 3) the chat transactions may 
serve as indicators for the level of engagement. 

The initial results showed that performing a secondary task such as chatting during a video lecture: 1) 
may reduce the volume of notes taken by over 30%, 2) negatively affected the participant’s short-term 
memory of the video content, and 3) may change the notes by adding multiple sources including the chats 
into the notes on the video content. The complete data results will be reported at the conference. 


3. CONCLUSION 

The number of portable connective devices continues to increase in classrooms resulting in a need to 
understand the effects of using these devices while attending to lessons in the classroom. Also, as the number 
of online learning opportunities increase, so does the access to opportunities to perform multiple tasks during 
a lesson. This includes both educationally designed uses of these technologies for learning as well as the non- 
instructional student mediated use of these technologies. Quantitative results indicate that student’s 
short-term recall was negatively impacted with student-mediated use of these technologies along with the 
amount of information that was actually captured in student notes while chatting during a lecture. We are also 
performing a temporal analysis where we will be able to examine trends that emerge from across the different 
data strands. This type of temporal analysis could provide a further window into what is going on behind the 
numbers and help us understand why some information is retained while other information is not as students 
attend to media multitasking during lectures. We hope that this specific contribution to the field will help 
provide evidence that can inform the effective use of these technologies as well as lesson design and delivery. 
At the end of each trial, participants completed a short assessment to see what they retained in short-term 
memory. While long-term memory is the objective of any learning endeavor, we believe that short-term 
memory is on the path to long-term memory and must be understood in the context of multitasking during 
learning activities. 
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